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Optimization problems involving linear systems with retardations 
in the controls are studied in a systematic way* Some physical 
motivation for the problems is discussed. The topics covered are’ 
controllability, existence and uniqueness of the optimal control, 
sufficient conditions, techniques of synthesis, dynamic programming. 
A number of solved examples are presented. 
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1, Introduction 

Optimal control problems involving systems with delays in the 
state variables have been studied extensively and the difficulties 
encountered in such problems have been well documented [1, 2, 8, 15 , 
17, 25, 24, 27 and the bibliographies of 2, 24] . Recently, more 
sophisticated models with systems containing retardations in both 
the state and control variables have come under investigation [2, 4, 

6 , 7 , 12, l4, 17, 24], In [2] Banks and Jacobs presented the mathe- 
matical foundations necessary for the study of very general control 
systems modeled by equations of the type 

0 t 

x(t) = / x(t+s)d F(t,s) + / h(u(s),s)d g G(t,s) 

-T — T 

4 

where F. and G are Stieltjes measures. The purpose of this paper 
is to investigate the effect (from both the theoretical and 
computational points of view) of lags in the control variables. We 
shall attempt to do this in a number of ways, hut our aim always 
will he to point out the pathological differences between systems 
with delayed controls and those without. In order to isolate the 

l 

effect of delays in the controls, we shall consider only the simplest 
models with lags in controls, and ignore any retardations in the 
state variables. Indeed, the examples of section 7 below illustrate 
very well the novel behavior of solutions to optimal control problems 
with these types of models, 

• In section 2, motivated by models arising in current applications. 
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we formulate several different types of systems which appear to be 

of interest c Controllability of these systems is considered in 

section 3 where results involving controllability matrices analogous 

aye 

to those for non-delay linear systems presented,. In the next two 
sections the questions of existence^ uniqueness^ and sufficiency 
conditions for time optimal problems are considered in the spirit of 
[11] „ In section 6 we extend to our systems a synthesis technique 
due to Neustadt [21]. A number of solved examples are presented in 
section 7 . These fundamental examples^ governed by systems which at 
time t depend on the control at times t and t - h, axe intrinsically 
more complicated than those involving systems which at time t depend 
on the control only at time t - h and give rise to prediction 
problems. Finally ; the paper is concluded with a section concerning 
the applicability of dynamic programming techniques to certain cases 
of the systems under study^ including mention of a Riccati type theory 
for quadratic payoff problems. 

We have tried to present numerous examples throughout the paper 
in order to provide the reader with an insight in regard to limitations 


of our results. 
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2. Notation and Formulation of Problem s 

We shall denote by £ the real vector space of all p x q. 

matrices o If A e £ the transpose of A will be denoted by A*. 

PI 

We shall not distinguish a column vector from its form as a row 
vector since it will always be transparent which form is intended 
by the order of multiplication in any matrix operations,, 

In order to facilitate the discussion of several types of problems 
involving various different system equations some special notation 
is required. We denote by the system 


x(t) = Ax(t) + B Q u(t) + B 1 u(t-h) 


where A e £ , B„, B, e £ and h is a positive constant,' The 

mP CP 1 nm * 

1 

system 


x - Ax(t) + Bw(t) 

is denoted by 5/( A,B) where A e £ , and B e £ „ 

J v 3 ' nn* nr 

The term 'control means a triple (u^t^t-!) where u' [t^-h^t^] 

t 

-> R is a function and t^t^ are realTrumbers,, 

Definition 2„1 . Given TJ C R m the symbol denotes 

the system ,S^(A,B constraint 


u(t) e U, 


t e [t^h,^] 



k 


on the controls {u,^,!^}, t e R. 

• Definition 2.2 0 Given U C R™ and a bounded measurable function 

£ 

v q : [-h,0] -> U, we use S^^(A } B^B^) to denote the system 
with constraints 

u(t) eU, t e [t^t^ 



on the controls, {u,^,^}, t Q , e R where u t (s) = u(t-t-s), 
s e [-h,0] „ 

Definition 2,g 0 Given U C R m and bounded measurable functions 
v i : [-h,0] h> U, i = 0,1, we denote by (A, B Q , B^) the system 
S^(A,B q ,B 1 ) subject to constraints 


u(t) e U, t e [t 0 ,t r h] 


U t Q = V 


on the controls {u,t t^), t^,^ e R. 

In the problems considered in this paper we shall often take 
U = R m or U = K 01 , where id 31 is the unit cube , 

{u = (u 1 , 0< , 0 ,-u m ) e R m I u 3 "} £ 1, i = 1, . 00 ,m), in R m . Whenever, 
h, A, B q , B 1 are understood S/ 7± will be. used instead of 
5 ^(a,b 0 ,Bi), i = 1,2,3. 
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Systems of type Sf with v Q = 0 are models for physical 
problems where at initial time t there is no delayed control 
effect, hut after some tame + h there enters a non-negligible 
effect on the system, at time t by the control given previously at 
time t - h„ This is exactly the case which occurs in the study of 
lossless transmission lines when one reduces a linear hyperbolic 
partial differential equation system with boundary controls to a 
linear differential-difference equation of neutral type in which 
control terms u(t), u(t-h) also appear linearly [14], 

Day and Hsia [7] have recently proposed a modification involving 
delayed controls for a model [18] of a gas-pressurized bipropellent 
rocket engine. In addition to being of type $/ , this modified 
model also provides motivation for study of systems in which the 
kernel of and the kernel of are complementary subspaces, 

-type systems are also models for continuous stirred-tank reactors 
as studied by Bay and Soliman [24] „ Although the example studied in 
[24] is non-linear, linearization about a nominal yields a system 

which satisfies kernel (B^C kernel(Bj) (see section 4 below). 

3 

‘Problems with systems of type S/ with v^ = v^ = _ 0 are 
motivated by air traffic control models currently under study [26]* 
one such model has system equations x(t) = -<0(t,x(t)) + u(t-h), 
y(t) = q(t) - u(t), where & is a landing rate, q is a queing or 
scheduling rate, and u is a takeoff rate. These models also in- 
volve systems in which kernel and kernel are complementary 

3 

subspaces „ Systems of type Sf with v^ = 0 are of importance in 
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so-called "settling problems"; i.e., problems in which one desires 
to attain the equilibrium state x(t-^) = 0 in such a way that the 
system will remain at this state without further control if other 
disturbances are absent „ 

We note that all three types of systems defined above are quite 
different from systems such as those modeling remote earth control 
of deep-space satellites studied by Foerster [ 9 ] and others [10, 

12 ; 23 ] which contain only control terms with a delay (i„e. , Bq = 0 ) , 

A control [u,t will be called admissible for the system 

y^(A,B ,5^ (or Simply j^-admissible ) if u t [t^h,^] -» R m is 
bounded, measurable and satisfies the constraints detailed in the 
definition of , i = 1,2; 3. Given x Q ,x^ e R n and t Q e R, we 
shall denote by P., i = 1,2,3, the problem of finding an in- 
admissible triple (u,t ,?} with U = K 51 satisfying x(t; 

^ -'min {t.J {u,t is i^ 1 - admissible with 

x(t ± ; t 0 ,x Q ,u) = x 1 }, where x(-; t 0 ,x Q ,u) is the response (solution) 
of system ,££(A,B ,1^) con ^ ro -L u with x(t Q ; t Q ,x 0 ,u) = x 0 „ 

That is, denotes the time optimal problem from x^ to for 

the system 5^ (a, B q , B-, ) with U = The special case of problem 

p with v = v =0 wild be denoted by P°, Finally,' we shall 
denote by P the special time optimal problem as studied in [11] * 
i.e„, the minimum time to origin for the system _9%A.,B) with 
U = K 01 . 

Necessary conditions in the form of a maximum principle for the 
problem are a special case of the general necessary conditions 
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derived previously by the authors [2]„ Using similar proofs one 
can derive necessary conditions for the problems P^ and P^(P^). 
Use of these conditions yields that an optimal control {u^t^t} 
for problem must satisfy 


(2.1a) 



sgn [\lr(t+h)B 1 ]^ t e [t Q -h,t-h] 
arbitrary^ t e (t-h,t Q ) 
sgn [t(t)B Q 3, t e [t 0 /t]. 


if 0 g t - < h ; and if h ^ t - t^ then (u^t^t) 


must satisfy 


(2 0 lb) 



sgn [t(t+h)B 1 ], t e [t Q -h,t o ) 

sgn [t(t)B 0 + i)r(t+h)B 1 ] - , t e [t Q; t~h) 

sgn [i(t)B Q 3, t e [t-h,t]. 


■where ijr(t) = t] e 2 q?(t-t)A, and the vector q ^ 0 is an outward 
normal to a support hyperplane for the attainable set at time t 
passing through the boundary point It is understood that when 

a,b e the relation "a = sgn b" is to be interpreted using the 
same convention as in [11, pg. 50] « For the problem Pg one obtains 
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the corresponding necessary conditions from (2.1a) and (2. lb) by 
deleting the requirements in the first two lines on the right-hand 
side of (2.1a)j and the condition on the interval [t -.h^tg) in 
(2.1h). For problem (p ) one always has U § h so that the 
situation in (2.1a) never occurs. Thus the necessary conditions 
for problem are obtained from (2.1b) by deleting the require- 

ments on the intervals [t^-h^t^) and [t-h^t]. 

Any admissible control in problem satisfying the above 

necessary conditions for will be called an extremal' control 

for problem P_^ i = ± } 2 } J>. Evidently ^ when computing extremal 
responses (i.e,,,, responses to extremal, controls) what one uses is 
what might be termed an extremal "effective control " } i.e.^ v(t) = 
B Q u(t) + B 1 u(t-h) J t e [t Q .,t] where {u,t^t} is an extremal 
control. This v is easily computed from (2.1a) and (2.1b) or 


their appropriate modifications for problems 



3. C ontr ollab ility 


In this section we shall derive necessary and sufficient conditions 

ar 

for controllability of the systems as defined above. These 

conditions will be analogous to the well-known rank condition on the 
controllability matrix for systems J^(A.B). 

Definition 3.1 . The system J>^(A,Bq,B_^), i = 1,2,3, is controllable 
on [t^, t^] if for every x^,x^ e R 11 there is an J/'’ 1 - admissible 
triple {u^t^t-jJ such that x^J t 0 ,x Q ,u) = x^. 


Remark 3'.1 . We shall find that the necessary and sufficient con- 
ditions for controllability are actually independent of the interval 
[t^, t^J as long as t^ > t^ + h„ Hence one could define the 
equivalent concept of a "controllable system" in addition to a 


"controllable on [t^ system". For the systems 5/%A f B) it is 
well-known that these concepts (and others) are equivalent [ 11 , 19 ]. 
Since we are mainly interested in obtaining the form of the necessary 
and sufficient conditions, we shall not pursue that aspect of the 
development here. 

Let us denote by jsl £ x £ the usual controllability 

J nn nr n(nr) 

matrix ^(A,B) = [B, AB, .. .,A n-- B] . 


Theorem 3.1 . / A necessary condition that (A, B Q , B 1 ) , i = 1,2,3 
be controllable on any [t^, t^] with t^ > t + h is that 
[^(A,B q ), ^’(A,B 1 )] have rank n. 
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Proof : ^^(A.Bq^Bi) -controllable ~>^(A : (Dq^)) controllable => 
5£(A } (B q >B 1 ) ) has rank n => [5f(A_,B 0 ), ^(A^)] has rank n„ 


The above condition will be shown sufficient for systems 




2 hl 

and Sf whenever U = R , but a much stronger condition will be 

3 ... 

necessary and sufficient for systems ^ a Note that the condition 
does not depend on h, the lag size. 


Theorem 3° g » Let U = R. m 0 A sufficient condition that 

B^) and -S^(A ? B Q , B 1 ) be controllable on every 
with t L > t Q + h is that [5f(A,B Q ), &(A } B^ ) ] have rank n. 


Proof : It suffices to give the proof for the system ^^(A^Bq^B^) . 

We shall give a proof that is a slight modification of that given 

for the systems Sf( A,B) in [19]. The usual constructive proof 

(see [13]) using a special symmetric matrix can also be made. Assume 

that [ (A^ B q ) ; ^(A J B 1 )] has rank n„ Let [t^tj with 

2 

t^ > t Q + h and v Q be given for B^B^) « Define x q( v q) “ 


B-.V (t-t )dt and consider (x (v n ) ) } the 


1 0 


0 


0 V 


- ( t 1 " t 0 ) A t o (t 1 -t-h)A 
~-e / e 

v h 

2 

attainable set at time t n for the system Sf. (A, B , B ) with 

1 h 0 i 

x(t Q ) = x q( v o^ 321(1 u = ^ e R m [ i = l r .»,m}. The set 


j^ M (x 0 (v 0 )) .consists of all points z of the form z = 

V h (t -t-h)A t l (t -t)A 

/ e B 1 u(t)dt + / e B Q u(t)dt where u: [t^t.^] 

*0 -0 


,m 



n 


i M 

is ‘bounded measurable with |u (t)| § M. We claim that S20 (x c (v Q ) ) 

C. R n has dimension n. If not, there is a vector 1^0 such that 

M 

\z = 0 for all z e v x q( v q)).> or 

t l" h (t -t-h)A t l (t -t)A 

(3.1) X. / e B..u(t)dt + ?v / e B_u(t)dt = 0 

*0 *0 

for all bounded measurable u with ]u 1 (t)| § M, faking u = 0 

(t, -t)A 

on [t^t-^-h] in (3«1) yields Xe = 0 for t e [t^~h,t^]. 

It follows by the usual arguments that XA^B^ = 0 for 

K = 0,1,2, thus \5£(A, B q ) = 0 and Xe^^ =0 for \ e R. 

(t -t-h)A 

Use of this latter result with (3„l) yields Xe B-^ = 0 for 

t e [t t-j-h], It then follows that X[ j^(A, B Q ), S£(A } B^) ] = 0, 

contradicting the rank condition hypothesized above, 

M 

That the n-dimensional set 5& ( x q( v q)) is compact and convex 
in R n follows from previous results by the authors [2] . further- 
more, it is easily seen that _q/^(x^(v^) ) is symmetric about the 
origin in R n and hence must contain a neighborhood of the origin. 
Since 2j^ M (x 0 (v Q )) C_c/ M (x Q (v o )) we find that the attainable set 
j^(x 0 (v 0 )) at time t ± for S/ 2 with U = R m and x(t Q ) = x 0 (v 0 ) 
must be all of E* 1 , The conclusion of the theorem then follows from 
the fact that 

( W A 

Ssf(x Q ) = e {x Q -x Q (r 0 )} + s^(* Q (v Q )) 

for any x Q e R n , 
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We remark that an obvious modification of the above proof will 

shew that the condition of the theorem is also sufficient for con- 

2 

trollability of systems of type B^B^) where one has a 

boundary condition u, = v_ in place of u = v . As one would 

\ 1 % 0 0 

expect, if U is a proper subset of R 131 , then the condition of 
Theorem 3„2 is no longer sufficient for controllability (see examples 
7.3 ; below). An immediate consequence of Theorem 3.2 is that 
systems Lx = b Q u(t) + b 1 u(t~h) will always give rise to and 

Sf type systems which are controllable. Here L denotes the 
usual real scalar n^ order differential operator with constant co- 
efficients, Lx = + a + o.o + a_x, 

3 n-1 0 


Remark 3.2. In a recent note [5] Do H. Chyung considered the eon- 
trollability question for systems of type Sf . He obtained as 
necessary and sufficient for controllability the condition that 
[5f(A,B 0 ), be of rank n. Note that from this con- 

dition one might suspect that lag size h could affect controllability. 

- 

- * Vi 

However, it is not difficult to show that [ :gf(A, B^), Sf(A, e -xm B 1 ) ] 

has rank n if and only if [5f(k } B Q ) , 5f(A } B^) ] has rank n. From 

a practical point of view, use of the second matrix is more desirable 

-hA 

since it can be computed without computing e . 
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In practice when delays are small in a problem one sometimes 

chooses to ignore them and work with an approximate system obtained 

by setting h = 0 in the original system. For i = 12 the sys- 
i 

tern B 0 ;B 1 ) is thns approximated by the system S/'{k } . 

In connection with this approximation we make the following observa- 
tion. 

theorem 3.5 . For i = 1,2, S^(k } Bq+B^) controllable implies 

i/^(A, Bq,B^) controllable when U = R m o 

Proof ; Bq+B^) controllable => ^(A^Bq+B^) has rank n -> 

[Bq+Bi^a(Bq+Bi)^ . . „^a (®o + ^i^' ? ’ * * * B o ] ^ as 

rank n =i> [ Bq), -^(A,B-^)] has rank n, 

It is easy to give an example to show that the converse of 
Theorem 3.3 is not true, e.g., take B x = -B q . Indeed, even in 
situations where the approximation might seem more reasonable, con- 
trollability can still be lost by use of the approximation. 

Example 3*1 . Consider the system 

x(t) - fV(t) + au(t) + bu(t-h) 
y(t) = gx(t) + cu(t-h) 


where a,b, c,f, g,h are not zero. One finds that 
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[ if(A, Bq) } has rank 2 while det 5f(A, B^+B^) = 

2 2 

g(a+b) - fc . Thus by using the approximation one destroys eon- 

2 2 

trollability if (a-fb) = fc /g. For example, if a = 1, b = -g 

l/2 

and c = (g/f) ' (1-e ) where g/f > 0, one would probably not wish 
to ignore the lag h. 

We remark that the results of this section can be extended to 
systems with multiple delays and even to systems with cert 8. in types 
of time variable delays. For example, for systems with dynamics, given by 

v 

(3.2) x(t) = Ax(t) + Z B.u(t-h ) t e [t_,t ] 

i _ 0 u i 

with 0 = h Q < h 1 < « o . < h^ ■ and u(t) e TJ, t e [t^-h t^, one 

can modify the previous proof to obtain the following theorem. 

Theorem 3.^ . Let U = K m . A necessary and sufficient condition that 
(3.2) be controllable on any [t Q , t^] with t^ > t Q + h^ is that 
lSf(Aj B Q ), if(A,B 1 ),...,*'(A,B )] have rank n. 

As a corollary to this theorem we obtain a sufficient condition 

t 

for controllability which does not involve A. 

Corollary 3.1 . For the system (3.2) with U = R m and (vfl)m 5 n , 
a sufficient -condition for controllability on any [t ,t^] with 
> t^ + h^ is that [B Q ,B^, . . .,B^] have rank n„ 
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Once one has obtained necessary and sufficient conditions for 

1 2 

controllability of systems and in 

terms of a rank condition on a "controllability mat race } one should 
be able to prove many theorems for these systems analogous to those 
for the system S/ r (A. } B) which involve the usual controllaoility 
matrix „ We shall present one such result involving the domain of 
null controllability, the proof being developed in a manner similar 
to one in [19] » 

We define the domain of null controllability for 
by 







there exists an admissible triple 

with x(t x ; t 0 ,x Q ,u) = 0 L 


2 

In a similar manner we define for the set 


®0 ( V 


lx e E n | there exists an J^-admissible triple 

1 0 1 

(u^o^i) with u t = v Q such that 


x (\; vv u > = ° 


Kote that for a given U, ^(v 0 ) C for any v Q <, We shall be 

especially interested in the set q(0) , i.e„, v^ =0. 
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Lemma 3-'l , Suppose 0 e U C R m and A 
2 

If ^ (0) contains a neighborhood -/A' 

^(0) = R n . 


is asymptotically stable, 
of the origin in R n ^ then 


Proof ; 
is in 





Given x Q e R n , let t > 0 he such that x(t; OjX^O) = e TA x Q 

p ^ ^ 

C ^ Q (o ) 0 Then let (u^t^t^) be $f -admissible with 

tA ^ 

0 such that x(t x ; t ; e x^u) = o. Defining 


u(i) = 


0 

u(£- T+t 0 ) 


i 6 t~h,T] 

< 

| e (T,T+t r t 0 ], 


it is easy to show that x(r+t^-t^J O^x^u) = 0 which implies 

*0 € 2>l( 0 ). 

Lemma g~,2 . Suppose 0 e int U and [Sg{A } ~B q) , 5f(A } B^)] has 
2 

rank n. Then & Q (o) contains a neighborhood of the origin in R . 
Proof' Let - J2^~ (y rt ) denote the attainable set at time t., corre- 

— " r 1 U-j^ (J 

sponding to y(0) = y Q using the system 

y(t) = -Ay(t) - B x w(t) - B Q w(t-h) t e [0,^] 

w t = Of w(t) e U, f e 
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This system may be thought of as the system (A^B^B.^) with 

v Q = 0" run in reverse time. Since rank [$?(~A } -B^), $g(-A } -B Q ) ] 

= rank \_5£(A } Bq ) } Sf(A } 3^)], arguments similar to those in the 

proof of Theorem 3,2 may be used to show that Slf (0) contains a 

t l 

neighborhood of the origin in R n for t^ > h whenever 0 e int U. 
It remains only to show (0) C 2#\(0) for t > h. Since 

e £*1 (o) is of the form 

h <t - b k-a) 

x -, = / e [-B,w(s) - B w(s-h) Ids 

0 x . 0 - 

where w, = 0, one can easily obtain 
t l 

t A t l (t -t)A 

0 = e x + / e [B u(t) + B u(t-h)]dt 

0 ■ u 1 

with u(t) = w(t -h-t) for t e [-h,t ], yielding that x.. e 

Combining the two lemmas one obtains the following results. 

Theorem 3,3 , Suppose A is asymptotically stable, 0 e int U, and 
[ Sa (A, Bq ) } A J B 1 )] has rank n. Then 0^(0) (and hence <^) 

is all of R n , 

Obvious modifications of the above arguments yield the following 


corollary. 



Corollary 3.2, Under the hypotheses of Theorem. 3 .5, we have 


= R n for- any v Q .. 

Remark 3-3. One can obtain a result similar to Theorem 3.5 for the 
systems with the condition = v Q replaced by 

u = v 1 . However, the rank condition of the hypotheses must be 
*1 

IxA. 

replaced by the, in general, stronger condition " 5f(A } e~ B-j+B^ ) 
has rank n" . The reason for this change will be apparent after our 
discussion on the controllability of systems of type B 0 ’V 

which follows. 


3 

Although controllability conditions for systems 5^’^(A,E 0 ,B 1 ) 

can be derived from basic principles as was done above for systems 

and ', we shall make a simple observation about systems of 

3 

type which will yield the same results immediately by applying 

known theorems [11, 19] for certain non- delayed systems.' For 
3 

5^^(A,B Q ,B 1 ) on [t ,^1 and v ,v given, a straightforward 

3 

calculation shows that the response x(‘* t^,x^,u) to (A, B^, B^ ) 

satisfies 


X (V t 0 ,x 0 ,n) = x(t r h; t 0 ,x 0 ,u) 


where x is the solution to system _S^(A, e^^B^+B^) on [t n ,t 1 -h] 


0 > 1 


. -(t -h-t )A 

subject to x(t Q ) = x Q = e fU Sc () + e A with A = ACVV V t 
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defined, by 

*0 (t _t-h)A 

Vi-W'l ■ E i v o (t -V dt 

V 

t l (t-t)A 

+ / e B v 1 (t-t )dt. 

t n -h 


Therefore it is not difficult to verify that S/^ (A B B ) is 

h v 3 l y 

controllable on if and only if S^(A f e^^B^t-B^ is con- 

trollable on [t^t^-h], It follows that studying controllability 
■5 

of systems (A_, ) is equivalent to studying that of systems 

SS(A } e hA B () +B 1 ) . Since the matrix $?(. A } e hA B Q +B 1 ) is rank equivalent 
to ^(A,B 0 +e" M B 1 ) J we have the following theorems. 

Theorem 3.6 , A necessary condition that B^B^ be con- 

trollable on any [t^t^ with t^ > t Q + h is that ^(A J B o +e“' hA B 1 ) 
have rank n. 

Theorem 3.T® Let U = R . A sufficient condition that 

- 

^(a, b 0 , b i) be controllable on any [t^t^] with t^ > t + h is 
that ^’(A J B 0 +e“ hA B 1 ) have rank n„ 

Remark 3.^ ® ‘The rank of A,B +e“ hA B 1 ) equals n implies the 
rank of [.^(A^Bq), .^(A^B^)] is n } but not conversely (see 
Example 3,2 below). Thus the rank condition of Theorems 3.6 and 3.7 
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is, in general, stronger than that of Theorem 5.1. Furthermore, 
the dependence of the rank condition here on the lag size h is 
not illusory (see Remark 3°2) as the following example demonstrates. 

Example 5° 2 ° Consider the system 

x(t) = uy(t) 

y(t) = -Tix (t) + u(t) + u(t-h) . 

For h = 1 we find B 0 +e _hA B 1 ) = 0 while for h = 2 

ifCA^+e-^) = 
has rank 2. 

_ The above example also shows that the systems Lx = b Q u(t) + 
bi u (t“h), ^ u^_ = need not he controllable (see the 

comments preceding Remark 3.2) „ It is also easy to see that con- 

3 

trollability of y^A,B ^ is not, in general, implied by 

controllability of either i^(A,B ) or 5^(A,B ). 

That a result on approximation similar to Theorem 3.3 does not 

hold for type systems can be seen from Example 3.2 above. 

Finally, defining the domain of null controllability <3 ^(Vq, v^) 

3 

for i^CAjB^Bi) i n the obvious way, we do obtain the following 


0 2tt 

2 6 


In addition, [#(A,B ), if(A,B, )] 


analogue to Theorem 3 « 5 » 



21 


Theorem 3.8 , Suppose A is asymptotically stable^ 0 e int TJ, and 
C J% ’(A, Bq-i- e _ h ' / T3 1 ) has rank n„ Then,, = R n for any 
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4. Sufficient Conditions for the Special Time Optimal Control Problem 
In this section we prove sufficient conditions for problems of the 
form P-^ P 2 _, or P° where U s'K , the "unit cube" in E m (see 
section 2) and the terminal condition x(t^; t^x^u) = 0. Actually ^ 
in sections 4 through 7 we always take - 0 so t will be 
suppressed in the notation x(t® and in the notation 

[u,t t^} for an admissible triple. The sufficiency condition in 
this section is an extension of a sufficient condition given by Hermes 
and LaSalle [11^ pg„ 72]. The discussion is facilitated by introducing 
the concept of the s'et of reachable 'states at time t [11] for 
problems P^ V r ^ } and P°. We say that a point (or state) x e R n 
is reac hable at' time t g 0 in problem P if there is an admissible 
u for problem P such that 


(4.1) x = / e As Bu(s)ds. 

0 

We say that x is reachable at time' t a O' in proble m. P^ P° 
if there is an admissible {ip, t] for problem P^ P^, P° respectively 
such that 


A<a 

(4,2) x = / e [B q u(s) + B 1 u(s-h)]ds. 

The symbols } &^(t) } 0®(t) denote respectively the 

set of all states x reachable at time t in problems P-^, 
Properties of is?(t) have been carefully studied in [11] „ The 
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behavior of .^(t) and i^(t) is somewhat more complicated. In 
fact,, we shall see that some of the basic properties of &(t) 
simply are not true for -^(t) and -^g(t) without making special 
assumptions on and B^„ 

If x,y e R P , then we use <x,y> to denote the usual scalar 
product in R P „ If SC R P , then 8“ denotes the orthogonal comple- 
ment of S, i„e., = fx e R P | <x,y> - 0, y e S} 0 If M is a 

p X q real matrix, i„e 0 , M e i , then we reserve ker (M) and 
Ira, (M) for the kernel and image respectively of the linear trans- 
formation x |-» xM, x e R P , i.e., ker (M) = {x e R P ] xM = 0} and 
Ira (M) = (y e R-| y = xM for some x e R P } » The following norms 
will he used for vectors x = e R P S 

Hxli^ ~ max (lx 1 !, i = 1, ...,p} 

1 1 x|| = \/^7x> 

W = 5 I* 1 !. 

i=l ' 

T ^e also use the symbol ||m[|‘ to denote the matrix norm subordinate 
to the vector norm II oil on R p and R p i.e., 

= max {||xM|| J ||x|| w Si, x e R P } 

P 

= max { S jm J j = l,...,«i}, 

i=l 
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where M = (m. .) } i = lj.„.,p* j = 1^.,.^. The matrix norm |m| 
subordinate to the vector norm j°| is similarly defined and is 
likewise easy to compute. 

Some hypotheses which will be invoked to obtain various results 
in the sequel are now listed for future reference. 


(Hi) Matrix B has a left inverse and 

0 UJj 

C = B”^B, satisfies ![c!j S 1, 

OL 1 1 H co 


(H2) 

Hypothesis (HI) with |C| 

< 1 instead of jjcll^ ^ 1. 

(H5) 

For any 

*1**2 

satisfying 0 < t^ < tg 


% 





I \ 

-As 

qe 

B 0 |ds > / | 

i Tie” AS B 1 | ds 


whenever 

n e 

R U and tj j 

t 0. 

w . 

. o__ 
V 

Nib 

x e [ker (Bq)] 1 = Bn [B*], x 4 0. 

(h4-») 

’fi 

o 

IIV 

1^1, 

„n 

x e R . 

- 


Proposition 4-.T.' There is an m x m real matrix G such that 


B^ = B Q G if and only if ker ("B ) C ker (B^) . 


Proof" Evidently > B^ = B^G implies ker (Bq) C ker (B^) . Conversely ^ 
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ker (B Q ) C ker (B.^ implies [ker (Bq)] 1 Z) [ker (B-^] 1 or 
equivalently, Bn (B*) D Im (B*) . The existence of G- with the re- 
quired properties follows at once from the last inclusion and some 
elementary matrix operations. 

Proposition 4.2 . (a) (h 4» ) implies ker (B q ) C ker (B^) . 

(h) (h 4) and ker (B q ) C ker (B^) imply (h4* ) . 

(c) If S/{k } B Q ) is proper 1 , ker (B ) C ker (B 1 ) > and (h 4) 
is satisfied, then (H?) is satisfied, 

(d) If (H5) is satisfied, then 5^(A,B 0 ) is proper, and (h 4> ) 
is satisfied, 

' (e) (h 4' ) and (H2) imply (h4) „ 

Proof l Statements (a) and (b) are obvious. Suppose (H5) is 
satisfied. Then for 6 > 0, t\ e R n , T] j 0 we have 

t^+S t^S 

^ / he“ As B 0 |ds > ^ / | 'ne" AS B 1 | ds, > 0. 

- t l t l 

Hence there results 

|T!e“ AS B 0 J ^ | Tie“ AS B 1 J , t) e R n , s & 0 

and (h 4* ) is -satisfied. Evidently (H5) implies ~9 ^(A,Bq) is proper. 
^See [11] for the definition of a proper system ^’(A,B 0 ) . 
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Now assume (H 1 *-), ker (B^) C ker (B^), and _9^(A, B^) is proper. 
Observe that 

R n = ker (B q ) © [ker (B^] 1 = ker (B q ) '© 3m (B*). 

' y\ A 

Choose r] f 0, T] e R and define ty(t) = rje” •» Then i|r(t) = 
v(t) + |i,(t), where v(t) e ker (B q ) and p(t) e 3m (B*). This de- 
composition is unique and \i and v are continuous. Choose 
0 < t^ < t 2> then (£&) implies 

id(t)B Q | > !p(t)B 1 ] 

on [t^tg] with the possible exception of a finite number of 
points since 5^(A,B Q ) is proper. The assumptions ker (B q ) C 
ker (B, ) implies 

-L (t 

[t(t)B 0 | > 1 ilr(-fc)B 1 | a.e. on [t-^tg] 
and (H5) follows at once. 

t 

Suppose now that (Hh 1 .) and (H2) are satisfied. Then B-^ = B^C 
by Proposition h.l. If x e [ker B Q ]\ x j. 0, then |xBq| > 0.' 

Whence |xB i | = [xB Q C| ^ [xB | |Cj < |xB Q |, and (H4) is satisfied. 

Corollary 4-.1 . If 5^(A,B Q ) is proper, ker (B ) C ker (B ), and (h 4) 
is satisfied, then |r]e" At B 0 | > | , r\ £ 0, tj e R n for all hut 

a finite number of t on any compact interval. 



27 


Example 4,1, Let L denote the linear differential operator 

- -t ( n ) (n~l) , 

Lx = x v ' + a 1 + • « • + a.x 

n-1 0 

where a^, i « 0 ; ,<,. ; n-l are real constants . Consider the control 
system Lx = b 0 u(t) + 4jU(t-h) where b Q ,b are real constants. 
Since we refer to this example several times in the sequel we write 
this explicitly in the form .S^(A,Bq, B^) » Let 



Then Lx = b u(t) + b,u(t-h)- is equivalent to the system 

v “4* 

Sr. (A, B_ ,B, ) „ The condition that matrix have a left inverse is 

h 3 O’ 1' 0 

equivalent to b Q ^ 0, Hypothesis (Hi) is satisfied if b Q ^ 0 and 

\ 

S 1 ( Clearly her (B q ) C ker (B^) „ Moreover, b Q j- 0 implies 

i/(A,B 0 ) is proper, finally (h4) is satisfied if |b Q j > Ib^J . 

Let r(t) denote any one of the reachable sets at time t (i.e,, 
^(t), ^^(t), ^ 2 (t), ^°(t))„ Then T(t) is increasing if 
0 ^ t^ g tg implies r(t ) C r(t 2 ), We say r(t) is expanding if 
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r(t) C Int (r(t ) ) for 0 g t < t-. «. Let K denote the character- 
-L , JL o 

istic function of a set S C X. Define 

(4.3) u(sj t,r[) = sgn (s) + ^-A^+h)^^ t _ fa ] (s)] 

for -h g s g t and rj e R n ^ rj f 0 o When u(sj t^T]) is referred 
to with -h g s g t it is understood that we are referring to 
problem -P^. The corresponding u(s; t,T|) for problem P 2 merely 
requires u(s* t^ rj) have the form (4.3) for 0 g S g t and 
Uq(* j t j t] ) - Vq, In problem P° we do not invoke this symbol,, The 
notation qp(t^u) where {u^t} is admissible in P-^P^., or *3 
defined by 

w. t 

(4.4) cp(t u) = / e ~ S [B n u(s) + B.u(s-h)]ds 0 

0 u x 

'It is also convenient to take the following definitions^ 

(4.5) z(t,r)) 1 = <p(t,u(.; M)),‘ 

(4=6) g(M) = <ai,z(t,ri)>. 

Proposition' 4.3 . 3^?(t) and &°(t) are increasing. 

Proofs The statement concerning i^(t) is obvious. Note that 



t S h is merely i^?(t-h) for problem P with system 
y(A } 3 Q +e~ Ah B 1 ) . For 0 g t g h, ^°(t) = (0}, so 0°(t) is in- 
creasing „ 

Proposition 4 * 4 * &^(t) , ^ 2 (t) are both increasing if any of the 
following three conditions is satisfied: 

(1) her (B q ) C ker (B^) and (HI) is satisfiedo 

(2) ker (B q ) C ker (B^) and a matrix G satisfying the con- 
ditions of Proposition (4*1) also satisfies ||g|| ot - 1° 

(5) (h 4 t ) is satisfied* 

Proof* We shall prove only that J^(t) is increasing is implied 
by (2) or (3)„ The remaining situations are similar* If 
ker (B Q ) C ker (B^), then an ra x m matrix G exists satisfying 
B, = B~G (Proposition 4*1)* If (2) is true we may take ||g|J g 1* 

X v ^ 

If p = cp(t^u) and 0 < t 2 - t^ < h, then w(t) = -Gu(t-h), 

^ t g tg is measurable and satisfies 

IKt)!^ g x 

B Q w(t) + B^ (t-h) = 0 

on [t^t^]* Define u J [ -h . ] -> K™ by the conditions 

u l|[-h,ti] = u, and uj ^ Then = tp^u) = p e 

i^(tg) and we infer that i^(t) is increasing* 

• Suppose that (h 4' ) is satisfied* Choose t^ 


and tg such that 
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0 S s tg and pick t] e R n , T] ^ 0. There axe support hyperplanes 
TT^ and 7Tg^ to and respectively with outward 

normal rj,. Thus there exist e .^(t^) and p 2 e .^(t^) such 
that CrijP-^ § <t],q>, 1 e ^(t 1 ) and <r^p 2 > § <n - q>, 1 g ^(tg) . 
Hence Cr^p^ - gCt-^T)) and <i].,p 2 > = g(t^i])„ Now g(t^n) as 
defined in (4,6) can be written in the form 


(^•7) g(t,T]) = / |iie“ A ^ S+ll ' ; B :L [ds + / ! Tje~ AS B 0 + &s 


+ / {Tie" S B jds, if t ^ h 0 
t-h 


TVr 

Prom (4„7) and (h 4' ) one deduces that (t, t)) g 0, t s h and 
t 1 — » g(t,T(), t ^ h is nondecreasing c Therefore, if t^ & h, then 


' H l n - ^ € ^1 <Q,q> ^ g(t x ,ij)} C = (q € R n ] <q,q> g g(tg,T))}, 
and since (t-. ) = D H, and &L (t 0 ) = 0 H 0 we have 

1 1 V° ^ 1 2 - ^0 2ti 

^(ti) C ^(t 2 ) for t^t^h. If 0 g t x , t 2 ^ h, then 

x ) C j^(tg) is clear, The fact that .^(t) is increasing 
is now a simple deduction,, 

It is easy to construct examples that show that (1), (2), and 
(3) in the preceding proposition are in general independent, We 
give below two examples showing that the conclusion of Proposition 
4,4 need not be true if some of the assumptions of Proposition 4,4 
are dropped. 
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Example 4.2 , Consider the scalar control system x = x + u(t) + 
Ku(t-l) where 1 - 2e < K < -e„ Using this system with problem P^ 
we see that 

g(t,l) = / |K[e“^ s+1 ^ds + J | e -S + Ke“^ s+1 ^jds 
-1 0 

t 

+ / e~ S ds, t > 1. 
t-1 

Since = [-g(t,l), g(t,l)] is a. compact interval and since 

•^| (t, 1) < 0 for t > 1, it follows that J^(t) is not increasing,. 



problem P ^ with v Q = 0„ Then her (B q ) $£ her (B^) and .^(t) 
is not increasing. For example, for t > 1 define p(t) to be 
max {x e b| (x,0) e i3? 2 (t)) 0 Then 


t 

p(t) = / e“ s ds 
t-1 

so that p decreases for t > 1, 

Proposition' 4„5<, ^°(t) is expanding t § h if and only If 

• 3 

^W e ' A \> is proper. 

Proof ” This follows at once from [11, pg 0 73] and the remark in the 



32 


proof of Proposition 4.3 0 

Proposition 4 0 6 0 if (H3) is satisfied, then j^(t), i = 1,2, are "both 
expanding 0 Moreover , if ker (B q ) C ker (B^) and i^(t) or (t) 
is expanding, then Sf{. A,B^) is proper. 

Proof , Mote that ^(t) , i = 1,2, are increasing (Propositions 4„2d 
and 4.4). Choose t-^tg satisfying 0 C tq < tg, Pick qe '(t) C 

i^pftp) « If q j£,Int (^^(t )), then q e Bd(ii?q(tg) ), the Boundary 
of J^q(tg) . Consequently, there is an q ^ 0 which is an outward 
normal to a support hyperplane for -^(tg) through q; i 0 e 0j 

<q,p-q> go, p e ^(tg). 

The point q has the form q = <p(tq,u) where [u,tq] is admissible 
in P 1<) A function Ug° [-h,tg] -s-K 111 is defined by 


^00 s 


u(t), -h g t g tq 

sgn [qe“ At B Q ], tq < t g tg 


Then {xL^jt^} adm i ss; j-^ le in Pq" P = cp(tg,u_), then 
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* / The AS B q | - | Tje^Bj ]ds 

*1 

> 0 

by (H3)o This is a contradiction. Hence ^ (t) is expanding. 

The same proof works for ^L(t). 

Now suppose ker (B^) C ker (B^) and i^(t) is expanding,, 

If is not proper, then there is an t) ^ 0, q e R n such 

that t]e Bq = 0, and consequently qe - = 0. Now the control 
function u^ = 0 has the form u(s* t, Tj) (see Equation 4.3). 

Hence 0 e Bdi^(t), t > 0 so that i£^(t) is not expanding. 
Analogous reasoning holds for the ease where ^(t) is expanding. 

It will be pointed out in section 7 when some solved examples 
of problems of type P^, Pg, P° are presented that hypothesis (H3) 
cannot be omitted and still obtain j^(t), i = 1,2 are expanding. 
Indeed, as we shall point out in the discussion of those examples, 
the hypotheses of the first part of Proposition 4.6 cannot be 
weakened,, and there does not appear to be an analog of Theorem 17.2 
in [11] „ The sufficient condition of Hermes and LaSalle [11] can now 
b'e stated. 

Theorem 4.1 . Let r(t) be any one of the reachable sets at time t, 
^(t), ^(t), iE,(t), i^°(t), If r(t) is expanding, if [u,t] is 
an extremal control for the corresponding problem, and if 
x(t* x q,u) = 0 , then [u, t) is a time optimal solution to the problem 



associated with P(t) . 

The proof of this theorem is obvious,, Of course^ the result is 
not of much interest without computable criteria for showing P(t) 
is expandingo Propositions 4.5 and 4.6 in conjunction with 
Propositions 4.2^ 4.3_, and 4.4 give us such criteria. 

Example '4. '4 . Consider problem. P^ with A = 

and = ( 1 ) * ■ Then is not proper ^ and it is easy to 

see t) is expanding. Moreover ^ (H3) is not satisfied. 




35 


5 . Existence and Uniqueness - for the Time Optimal Contr ol Problem 
It is easy to modify the uniqueness theorem in [11, pg. 69] to 
apply to problems P^, i = 1 , 2, 3 * Ti.ro admissible controls 

are regarded as equivalent if = t 2 and 
u x (t) = u 2 (t) a.e. on [-h,t^]„ An admissible control [u, t^} for 
Pp is said to be bang-bang if ju(t)| = m a„e„ on [-h,^]. 
Similarly, an admissible control [u, t^} for P^ (respectively 
P^) i s bang-bang If the above condition is satisfied a«e„ on 
[G, t^] (respectively [O^t^-h]). The following extension of a 
result in [11] is obtained. 


Theorem 5°’1 » If (u,T;} is an optimal solution to implies 
(u^t) is bang-bang, then there is at most one optimal control for 
problem P , i = 1,2,3, 

Proof ” One merely supposes there are two optimal controls (u^, *t], 
rn Pr 0 ^! 021 P^ which differ on a subset of [-h,t] of 
positive measure,, Then x(t; x Q ,u 1 ) = = x(t; x tig). If we 

‘ V ' - TO ‘ I ^(*0 + V 2 (t) 

define v l [-h,t] -» KT by w(t) = , then {w,t} is 

admissible in P. „ Moreover, it is clear that x(t° x ,w) = x and 
x 0 1 

(w, t] is not bang-bang, This is a contradiction. 

One never obtains uniqueness of the optimal control problem 
if 0 g t ^ h since the control (u, t} is not effective in 

for t - h £ t S 0, For this reason when we discuss 
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uniqueness of the solution to problem we assume t a h„ This 
is only a minor point and the situation 0 S t % h can essentially 
be treated as in [11] 0 

The next result is a reformulation of a general existence 
theorem obtained in [2] . Actually, problem was not discussed 

there, but the existence theorem easily extends to this situation. 


Theorem' 5.2 . If there is at least one admissible control (u,t^} 
for problem P ± satisfying x(t i ; x u) = x ±} then there is an 

4 " _ 

optimal solution to problem P^, i = 1,2,5. 

Proposition 5f3 » There is at most one solution to problem P^ if 

i/(A,B 0 ), V(A,B 0 +e" All B i ), and V(A, Bj) are normal (see [11] for 

the definition of normal). The statement of uniqueness holds for 

problem P^ if 5 /? (A,B 0 +e“ Ail B 1 ) and 5 /? (A,B q ) are normal, while 

for P-, the normality of J^fA.B +e -Ail B-, ) suffices. 

5 0 1 

Proof , We consider only problem P^o Clearly, the necessary con- 
dition (2.1) and normality of the three systems imply that the 

i 

hypothesis of Theorem 5.1 is satisfied, 

4- ' ' r ' 1,1 rr r " T " 

The problems P^, i = 1,2,3 were formulated so ‘that the admissible 
controls were in the class of Lebesgue measurable functions. The 
results in [2] when specialized to the present situation reveal 
that we could just as well have restricted our- attention to piece- 
wise continuous controls. 
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-Ah 

If A, and e are known, then computable conditions 

assuring the normality of B Q ), (A, B^+ e “ ) , and ^(A ; B^) 

-Ah 

are given in [11] „ In general, e is difficult to determine so 
we would like to obtain conditions that can he directly computed • 
from A,B^, (in this connection it should he observed that, in 
general, the normality of any two of the systems 5^(A,B Q ), 

SS( A , BQ+e"^), V(A ,B^) does not imply the normality of the 
third,. For instance in Example 3. 2 and S/(A } B^) are 

normal hut Jy^(A, B^+e~ B^) is not normal if h = 1 0 ) Some results 

are possible in this direction For example, let us consider the 
control system .5^ (A, B^, B^) discussed in Example 4.1. Along with 
the differential operator L in that example we consider its ad- 

-X- 

joint L given by 


t* ( n ) 

L x = x v ' 


- a 
n-1 


(n-1) 


(-i)V 


It is now assumed that |h [ + [b-J ^ 0 in Example 4.1„ 

Proposition 3.4 . System .S^A^Bq) is normal if and only if h Q ^ 0. 
System Sf{k } B-^ is normal if and only if h^ / 0. If h = b^ ; 
and if Lx = 0 has no nontrivial solutions of period 2h, then 
SAA, B Q +e^\) is normal. On the other hand if b^ - -b^, then 
£f{A } B 0 +e "*^B 1 ) is - normal if and only if Lx = 0 has no nontrivial 
solutions of period h. 

Let 1(A) denote the eigenvalues of A, and let Re 1(A) denote 


the real parts of 1(A). 
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Proposition 5.5 . If |b 0 J < |b.J, Ee ^( A ) - ° } then 

^(A.BQ+e-^) is normal. 

Proposition 5.6 o If [b Q | > and if Ke ? “( A ) - °j then 

ytA,B 0 +e~ iUl B 1 ) is normal. 

Propositions 5° ] K 5.5 and 5.6 are pretty clear, so we will only 
indicate the proof for one of these (Proposition 5.5). If b Q “ 0, 
then Proposition 5.5 is true. Thus suppose b Q ^ 0, Suppose 
^(A,B 0 +e" Al3 B 1 ) is not normal. Then there is a nontrivial solution 
ijr of L*x = 0 such that 

■b 0 t(t) + h^Ct+h) = 0. 

An easy induction argument shows that 

(5.1) *(t+Kh) = („i) K ^_£^ t(t) 

K= 1,2 3. ... . Since f(t) is nontrivial there is a sequence t 

, K 

such that , t.. — > <» as K -» co and i[r(t ) 0 as K -> co. This con- 

jx K 

tradicts the assumption that He l(-A) = -He 1(A) g 0. This proves 
Proposition 5.5. 

Example 5Vl ° Let Lx = x + a^x + a^x, h = 1, b Q = 2, b^ = 1, 

- 2 2 + 
a x = 2 log 2, and a = (log 2) + tt „ Then 1(A) = {-log 2 - iri), 
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and 


Tif(t) = exp[t (log 2 _ 7/1) ] 
is a solution of L x «= 0 satisfying 
2t(t) + i|r(t+l) = 0 

and for this system ^(A^B^-i-e"^^) is not normal. 

Example 5.2. Using the same notation as Example 5«1, consider the 
control system 

L*x = u(t) + 2u(t-l) . 


Then 


t(t) ~ exp (ini - log 2)t 

t . ^ 

is a solution of Lx = 0 and 


i(f(t) + 2i|r(t+l) = 0, 


and for this system 


$/{ A^Q+e*^^ 


is not normal. 
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6. Synthesis' for the 'Special Time Optimal Control' Problem 

Peustadt's method of synthesis [21] can be extended to cover 
problems P_^, P g , P°. Some rather restrictive assumptions are re- 
quired for problems and P^. It is assumed in our discussion 

in this section that x = 0 (x^ is the "target" ). The 
development will be carried out only for problem P , but if the 
arguments are suitably adapted problems ?,-> and P° csn also be 
treated. The validity of Neustadt* s approach depends on the follow- 


ing condition for problem P° If {u,f} is an extremal control 
for problem P satisfying x(tj Xq,u) = 0, then [u, t] is an 
optimal solution, to problem P. Ueustadt [21] assumed that the 
system S/{ A,B) was normal so that the above condition turns out 
to be satisfied by the sufficient condition in [11, pg. 72] „ For 


the problems we are studying, however, the optimal control (u,t) 

may be unique where all three of the systems 

5^A,B 0 +e'' iUl B :L ), _9TA, B^) are normal and yet J^(t) can fa.il to 
be expanding (see Example 7°1) so that the analogous sufficient 


condition for problem P^ could fail. 

Recalling the definition of z(t,q) in Equation (4.5), we can 
obtain the following proposition. 


Proposition 6.1 . Let the following conditions be satisfied; 
her (B Q ) C her (B^, -S^A^), ^A^+e"^), S^(A } B ± ) are 

normal, and (H4). Let S = (q € p n j <q,x^> < 0}. If the optimal 
control (u, t} exists for problem P^, then it has the form 
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(4.5). Any vector rj e S which maximizes the time t for which 
<T b z (' t ; T l) > ~ -<T], x q> may he used in (4,5) to obtain the optimal 
control (u,t) = {u(*,t,T]), t}„ Conversely, if rj defines the 
optimal control (u,t) by means of (4.3), then it maximizes the 
above time t. 

Proof o Kote that g(t, rj) defined in (4.6) can be written in the 
form (4.7) if t ^ h, and if 0 ^ t ^ h we get 

(6.1) g(M) = / | ije“ As B | + |r[e“ As B Jds. 

0 

Hence (4.7), (6.1), and Corollary 4.1 imply that (t,T)) >0 so 
that t 1 — > g(t, T)) t S 0 is strictly increasing. The function 
CM) M g(t,Tj), t ^ 0 } i) e S is continuous. For rj e if 1 , ‘T| ^ 0 
we have that 

(6.2) <ri,z(t,T])> > <n,X>, x e ^(t), x^ z(t,T|), 

by the normality assumptions in the proposition. Define 
f. [0,») X S X R n -» R by the equation 


f(t,11,X 0 ) =<n,z(t,Tl) + X Q >, 


and define 
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H a £ Ctj € R n | fi^O. <n,-x A > = max 1_ <t],y>}. 

0 0 ye^flO 

The set H Q is convex . Observe that 

(6.3) f (0,Ti,x 0 ) < 0, r) e S 

whereas 

(6.4) f(t,T],x 0 ) >0, r) -e S s \ Hq. 

The assumptions of the proposition imply that i^(t) is expanding . 

Hence Theorem 4.1 and relation (6.2) assure ns that 

(6.5) *(M,* 0 ) = 0 

implies that t = t if T] e H^. Hence using (6.3), (6.4) and the 
last remark it is seen that (6.5) defines t implicitly as a func- 
tion of ri, for t) e S. We denote the function so defined by F. 

Then F is continuous and F(r() = t, rj e H Q , and t > l(r]), r| e S\H Q o 

The purpose of the observation in the above proposition is to obtain 
a method- for finding a vector tj which.- can be used in (4.3) to determine 
the optimal control. It is easy to see that g is a C^" function on 
([0, »)\(h}) X S by direct computation in formulas (4.7) and (6.1) 
using standard results on the differentiation of Lebesgue integrals 
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involving parameters [20, pp„ 216-217] and the normality hypotheses 

of Proposition 6.1. Hence if q e S is such that P(tj) £ h, and 
’rV 

(P(T]),rj) f 0, then the implicit function theorem tells us that 
P is continuously differentiable on a neighborhood of Using 

the fact that under the assumptions of Proposition 6„1 ^^(t) is 

expanding (so that the sufficient condition, Theorem 4„1, applies 
to P^) and the above remarks, the gradient technique for determin- 
ing the maximum of F on S can be applied to Problem P^ Q We do 
not carry out the details here, but refer the reader to Weustadt' s 
paper [ 21] „ 



7. Examples « 

In this section we solve some examples which illustrate the 

strange behavior of solutions to problems of type P^, P^, P° 0 All 

of the examples are two dimensional,, Since we would like as much 

as possible to avoid using superscripts and subscripts, we shall 

1 2 

agree in this section that (x,y) = (x x ) . 

Example 7.1 . The system equations are 

= y 

= u(t) + u(t-l) . 

B = B, = ( ? J , h = 1. Here we consider 

0 1 \ 1 / 3 

with boundary conditions , 

(7.2) (x 0 ,y 0 ),u) = y(t x ; (* 0 ,y 0 ),u) = o. 

2 

It is not difficult to see that given any ( X Q_>y 0 ) e E there 
is an admissible {u,^] satisfying (7.2) „ Hence there is 
(Theorem 5.2) an optimal solution to problem P^. Proposition 
and Theorem 5.1 assure us that the optimal control (u, t) is unique 

if t £ 1 and if 0 ^ t < 1, {u,t} is unique where it is effective, 

i.e., on [-l,t-l] sind [0,t]„ The necessary condition (2.1) when 


C7.1) 


x 

o 

y 


Thus A = 


0 1 
0 0 


a problem of type P 


applied to this problem yields 



-1 I t s t - 1 


( 7 - 3 ) 


sgn [i{r (t+1)] 


u(t) 


undetermined 
sgn [i 2 (t)] 


t - 1 < t < 0 
ostst 


if 0 ^ t ^ 1, and if t > 1, then 


( 7 . 3 ') u(t) 


sgn [* 2 (t+l)] , 
sgn [t 2 (t) + t 2 (t+l)3, 
sgn [f 2 (t)] , 


-1 I t < 0 
OSt<t - 1 
t - 1 s t s t 


where ijr = Of ) is a nontrivial solution of the adjoint equa- 

* 2 2 
tion ijr = -iJrA 0 Hence i|r (t) = pt + 5 where i|r (t) is not 

identically zero,. Along with the optimal control (u,t} we con- 
sider the effective optimal control {v,t} where 


( 7 .^) v(t) = n(t) + u(t-l) 0 


With problem P for system ( 7 „ 1 ) and boundary conditions (7,2) we 
consider the auxiliary problem P with system S^(A,B Q ) only with 
the restraint set changed to [- 2 , 2 ]. The synthesis for this 
problem except for an obvious scaling factor of 2 (i.e., the 

switching curve is x = -y fk } y § 0 and x = y /4, y ^ 0) is 
described in [22] 0 if {w, t} is the optimal solution to the 
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auxiliary problem P and if w is expressible in the form 
u(t) + u(t-l) with {u,t} admissible in P , then {u, t} is the 
optimal solution of P^„ Thus can be considered solved if 

0 S t § 1, Figure 1 shows the reachable set i?£^(l) and the 
synthesis in case (Xq^q) e ° 


Figure 1= 

Thus we now assume that (x ^y^) £ 1^(1) so that t > 1„ Here 

the situation is a good deal more complicated since the above w 

is no longer expressible in the required form. It is noted from 

(7.5 J ) and (7=4) that the effective optimal control has v(t) taking 

only the values in the set {-2,0,2}, 0 S t S t. For brevity let 

us denote the optimal trajectory issuing from (x^y^) by (x,y). 

Then" (x(t),y(t)) can reach (0,0) only along one of the two 
* 

curves 

S + ° x = y 2 /4, y ^ 0, 

S : x - -y 2 /4, y > 0„ 

If p = 0, then 6^0 and from (7=3') we see u*(t) = sgn (8), i.e„. 
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there is no switching* Hence 5 > 0 implies (x rt ,y ) e S and 
SCO implies (x 0 ,y Q ) e S_„ Conversely, ( x 0 ^y o ) 6 S + U S__ implies 
P = 0* If ( x o-’ y O^ ^ s + U S_, then p f- 0* It is not difficult 
to show that p > 0 or p < 0 accordingly as (x^, y^) is to the 
right of S + U S_ or to the left of S + U S_<, Let -5/p he de- 
noted hy If one finds v(t) = u(t) + u(t-l) using (7.3’)^ 

then it is clear that both v and u will be known completely if 
the disposition of the points - ^ + & 3 (3, ^ + (3 relative to 
[0,t] can be discovered* Now the boundary conditions (7.2) impose 
additional conditions on t and (3. In fact with u = u and 
t^ = t (7»2) reduces to 

t 

(7-5) x n = / sv(s)ds 

0 

' t 

y n = ~-f ^( s ) ds ° 

0 

By a systematic and laborious enumeration of the possible positions 
of - ~ + f3, £3, i + £ relative to [0,t] it can be shown that 
(7*3') and (7.5) uniquely determine t and p> as functions of 
(x 0 ,y Q ) jk S + U S « In principle at least the determination of 
t(x QJ y 0 ) and ^fx^y^) represents no difficulty, so we shall only 
describe the results* However, it must be pointed out that when 
the possibilities are exhausted our calculations revealed the 
following* There corresponds to each e r \ 7^(1) exactly 

one extremal control satisfying the boundary conditions (7«2). 



Hence an extremal control satisfying (7*2) must "be time optimal,, 
(.It mil be seen momentarily that & (t) is not expanding, so 
Theorem 4.1 does not apply.) 

Let D (respectively D ) denote the open region to the 
right (respectively, left) of U S . Sets D , i = 1,2, ...,7 
are defined by the following relations S 


D n 


- e d rI x o - 2 “ ~ ~§ 


D„ = 


14 = 


^v y 0 ) 


C ( V y o } 


6 D r! “ 2 ~ y n < °> 


■1 y 0 


7 o* < 1 y o y o 

s x„ g - + 




1 y 0 y 0 


2 2 
3 3 „ , y 0 


6 V y 0 - °> 2 “ “2 - TT < x o s 5 “ ff y 0 + * 1 ? } U 


1 y o y o 


3 y o A 


f fx 4 y ) e D I y ^ o — - —id + - ^ x ^ ~ — 1 

U 0 ,J V ^R 1 y 0 - ' 2 Tf *K - o " 2 


D, = 




y 


0 


y o y 0 


eD fi l *o‘- a .-r‘*o < -T--E 1 ' 


D, = 


(CvV 


y o . y o 


y. 


and x *1-1° !° } 

0 _ 2 


3V = 


! W 




eulogy £ 2, 2 _ ~ y + 4 £ x sf--2}U 
R 1 J 0 J 2 2 J 0 8 0 2 4 ; 

2 

. 5 y 0 

e D v £ 2 x ^ — h) 

R 1 > n 9 “ V 


0 " 2 4 


V 




I 3 y 0 

e V y 0 s °> x n * % - 4 } U 


' 0-2 4 


Yr 


»V y o ) * d rI y o s °> x o s i - 2j o * - >• 


We define D i = 1,2, ...,7 by symmetry through the origin, i.e„. 





D ~i = “ D i = “(vV e - D i } > 1 = Then 

r t i 2 

D . U (J V ± U [S + -U S ] is R } and ^(l) = cl (2^) U 

1 _J J Li=l 1J 

cl (D where cl (E) denotes the closure of E<, The regions 
D _i> V 3 +> S -' 1 = are depicted in Figure 2. 


Figure 2 a 


The following formulas obtain for t and 


y o + ^ 2y o + - 3x o 


> (x 0 ,y Q ) e D 1 


■^p. i yp i 

' ^0 * ^0 " + 2 ’ (X °' y ° € Da 
1 + t 1 + 5 V\ * + 2y 0 - 2, (* 0 ,y 0 ) * d. 




x o y o 


-K-T* 


-1*5* o + + 2te o > ' Vo> € ®5 


'o 1,1 


+ 2 + 2 ^ 6(x 0 + V* 0 / ( V y 0 } e D 6 


-§ + v 1 + 2(x 0 + y^4) , (x 0 ,y Q ) e d ? . 
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-V > (x o> J o } 6 s - 


Vo> = 


- -§ > ( x 0 ’ y o' > S S +> 


y + 2t 


p Vo> - 


4 

3 

y + 4t 

- 2 

4 


y ff t. 


8 

3 

v + & 


2 

i 

y o * & 

+ 2 

6 


y Q + 4t 

- 2 


' 0 '" 0 ' 1 “ 7 


. e D 2 


-0"0' 3 


> 6 *V 


; ( x o’ y o^ e 


» (Vo 1 e d 6- 


It is noted that if H D^. ^ 0 for some i,k' = 1 } 2 } oo^ } 7, then 

there is still no ambiguity in the formulas for t(x QJ y 0 ) and 

(3 (x Q ,y 0 ) o In order to complete the definition of t and (3 on all 
2 

of E we merely take advantage of the symmetry in the problem to 
observe that t(x Qj y 0 ) = t(-x Q ,-y 0 ) and p(x 0 ,y 0 ) = p(-x 0 ,-y Q ) if 
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(x^y^) e D^. We note that t is not continuous at points on 

S US and on D D cl (D )„ However, at every other point of 

D both t and f3 are continuous „ 

R 

Wow to see the nature of the optimal trajectories we describe 
the optimal effective control v(t) = u(t) + u(t-l) if the initial 
data (x Q , y Q ) e D^, i = 1,2 , . 7* We use v. to denote the 

optimal effective control defined on [O^tfx^^y^)] if (x^y^) e 
Ih, i = 1,2,...,7. Of course, if (x^,^) e D ± , the n the optimal 
effective control is i = 1,2, 00() 7, The formulas for 

are as follows l 



0 =S t p 

— } 

p<tst 

0 S t * t - 1 
t-KtSp , 
P < t g t 

OS.tS-g+P 
J+P<tst-1 
t-Kt^p 
p < t S 1 

1 < t ^ £ + p 
i + p < t g t 
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0 ^ t ^ p 
p<ti t 




y 5 (t) 


'-2; 0 ^ t £ - ~ + p 

o , -~+p<tgp 

+2 , p < t § 1 

0, i < t s J + p 

+ 2 , ~+P<tSt 


) 



~-2 } OStS-i+p 
0 , .j+p<tst-l 
-2 ; t - l<tsp 

0, P<tSj + P 

+2, g. + p < t § t 


v 7 (t) 


'- 2 , osts.i+p 

o, *i+P<U^p . 
+2., J+P<tst 


If (x Q /y 0 ) e E +l u B +2 U D +i) _ U D + ^ U D + ^ ; then the optimal 

trajectory, (x(t) ; y(t) ), beginning at (x^y^) can he described 

in a simple geometric fashion. If ( x q>Yq) c U D ^) = 

then this description is given in Figure 1. Evidently, if 

(XqjJTq) e D^, then v^(t) switches from 0 to +2 as (x(t),y(t)) 

crosses S . Moreover, in all cases where (x„,y ) c the last 

+ 3 ' Cy 0 R 
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switch occurs as (x(t),y(t)) crosses S . Wow let C 01 = 

1 2 + dl 

[ (x,y) [ x = ?r ~ ; ~2 g y g 0] (note C 21 is a segment of 

Bd (.£^(1))). Then if ( x 0 *y 0 ) e Dg, the point (x(t),y(t)) coasts 
to Cg-j and then the synthesis for i^(l) obtains (Figure 1). Let 
curves Cg 2 , 0 ^ be defined as follows! 

7. * 2 

c 6i = { ( x ; y ) f x = g - it y + * -1 - y - 1}* 

_ ' — 2 

Cg 2 = C(x,y)| x = g--~-2g., “1 - y - 

2 

% « C(x,y)| X.-I+2,, -Uy^o}. 

If (x y ) e then the first, second, and third switches of v^(t) 

take place as (x(t),y(t)) crosses C^., i = 1,2,3 respectively <, 

,2 01 

Finally, define = {(x,y)| x = 2^ - y, y g -1} „ If (x Q ,y 0 ) e D 

then the first switch of v (t) happens when (x(t),y(t)) crosses C 71 . 
If (x 0 ,y Q ) eDgUD^UDgUD^, then by use of symmetry the optimal 
trajectories are similarly described using curves C . . = -C. .. The 
synthesis for ( x 0 ,y 0 ) € f> 2 u D 4 u D 6 U D 7 is silown in Fig 111 ’ 6 5° 


Figure 3 « 


For (x 0 ,y G ) s U the set of "first switching points" do not 



lie on a curve and the situation is too complex to describe geo- 
metrically. Some typical optimal trajectories are given in Figure 4 
for <x 0 ,y 0 ) e U D^. 


Figure 4. 

It is interesting to note that some of the optimal trajectories 
initiating In or Dg can come to rest on the x-axis for a 

positive time duration before continuing on to the origin. Tra- 
jectory A in Figure 4 shows an instance of this, hut this is not 
typical. 

In this example, ^(AjBq), and A, e ) are 

all normal (and all proper) and yet .^(t) is not expanding al- 
though ^-^(t) is increasing. The boundary of i^(t) for a few 
values of t is sketched in Figure 5. 


Figure 5„ . 


This figure clearly shows i?^(t) is not expanding. 
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An example of a problem, of the form 


is now considered,, 


Example 7. 2. The example considered here is exactly the same as 
that treated in Example "(.2 except that here we impose the con- 
straints 


(To 6) 



u - 0„ 

*1 


We give only a brief 
able set i^°(t) is 
S^B +6*^). The 


discussion of the solution to P°„- The reach- 

the same as j^(t~l) for system 

control system J^(A,B 0 +e“ A B 1 ) is given by 


( 7 . 7 ) 


x = y - u(t) 
y = 2u(t). . 


2 

Wow given (x^y^) £ R there is an admissible control {u,^} for 
P with system (7.7) such that the response of (7„7) to this con- 
trol satisfies 


x(t x ; (x 0 ,y 0 ),u)'= y(t x ; (x Q ,y 0 ),u) = 0. 


lence the same is true of (7«l) with (u, t +1} admissible in P°. 
this assures, us that an optimal control (u, t } for P° exists and is 
inique (Theorem 5.2, Proposition 5«5)o We note that if u(t) = 1 
in' (7«7) then we obtain the curve 
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s + : x = t: - 1 5 y * 

and if u(t) = -1 we obtain 

2 

S_. x = - 2^- ~ ^ } y S 0 . 

Figure 6 illustrates the synthesis for u using the auxiliary tra- 
jectories from (7„7)o 


Figure 6 0 

We let D (respectively D ) denote the open region to the 
xi ^ L 

right (respectively, left) of S + U S . For this problem the 
regions Ih , i = 1, 2 } . „ „ } 5 are as follows l 


D. 


y, 


i ‘ «V y o> € e kI x o s f - y o - v ! » 




D P = t( x 0 ,y 0 ) £ »Jo-y,--^ A ^ 


R 1 2 


8 " 0 


2 y o + T 


0 


y o y o 

and 


) , 



57 


D 5 = {( V y o } e D r 1 2 ~ T 


y o y o 


_ g X g 2 - J y + ° } 

4 0 2 *0 TT j 


y« y-. 


= C(x n ,y n ) e d r I y 0 S 2 and X n ^2-^.J}U 


0 ,J 0 

V y 0 


0 


~2 4 

2 


t^ y n) ^ R |y 0 ^ and 2 - | y Q + ^ g X() g 2 


y o y o 

“5 "nr 


} , 


5 „ , y o _ ^ 0 y o y o 


4 - dW e V x o * 2 - 2 y o + T » *o s 2 - 1 - 1 


The sets D ^ i = 1,2 ; ., S; 5 axe defined by symmetry as in 
Example 7 a l„ We have 


r 5 1 


r 5 1 

f? h 

1 

U 

U».i 

_x=l 


U [S + U S _] = E 


The regions D + ^, i = 1,2^..^ 5 are shown in Figure 7 . 


Figure 7„ 

Using the boundary conditions (7«2) and the maximum principal 

for P° one can show 

5 
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0 , -1 g t g 0 


u(t) 


-1 ; '0 < t g X 

+1 , X < t < t ~ 1 


o , t-ists t 


where 

‘(VV = 1 + \ + + % 0 * \ , 

> -( : r J ’ 7 o ' 1 = "Tf + "i - , 


if (x o ,y o ) 6 d r » By symmetry we have 

“ ^-VV' x< -*0> 7 0> * x( - x 0’- y 0 ) 

if (x 0 ,y Q ) e D^. The optimal effective control (v, t} for 
(x Q ,y 0 ) e D. is denoted by i = +l,+2, „ „ o } +p„ Evidently, 

y ± = ~v i = 1,2, „ „ 0 ,5o The following formulas for v. are 
obtained* 
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0 g t < X 
Ut<t - 1 
t - 1 S t < 1 

1 g t < X + 1 
X + 1 s t s t 


) 



0 g t < X 
X £ t < 1 

1 g t < t - 1 
t-lgt<X + l 
X + 1 g t g t 





0 g t < X 
X g t < 1 

1 g t < X + 1 
X+lSt<t-l 
t.istst 


} 


-1 , 0 g t < 1 

-2 , 1 g t < X 

V 4 (t) = 0 , X g t < t - 1 

-1 , t-lgt<x+l 

■ -+1 , X + 1 g t g t 



6o 


-1 




-2 


) 



+2 


) 


+1 




0 S t < 1 

X g t < X + 1 
H - 1 

t - 1 g t g t 


Figure 8 shows some typical optimal trajectories for 
( V y 0 ) £ D +t^ 1 Figure 9 illustrates some 

curious phenomena for this problem,. 


P° with 

3 

additional 


Figure 8„ 


Figure 9„ 


For example if (x^y^) = (-2,2) e D then the optimal trajectory 
to the origin is simply the arc pO of the curve x = -y /2 
connecting (-2,2) to (0,0) (Figure 9)„ However, a subarc pg 
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of arc pO is contained in Dg. Thus if one starts at a point n 
on the subarc pq, then the optimal trajectory does not follows arc 
nO to the origin, but will go off on a rather pathological trajectory 
(curve 7 ~ in Figure $)) } finally coming to the origin on an arc rO 

of the curve x = y /2 C Figure 9 also depicts what can happen when 

2 2 . 
x Q = +y Q / 2 „ For example, starting at point £ on x = y /2 the 

optimal trajectory is the curve cr 0 Note that cr in Figure 9 hits 
2 

0 at time t = ~ bounces down and then swings back to hit the 

origin at time t ~ 2„ Other variations of this type of behavior 

can also occur because of the boundary conditions on the controls in P°. 

5 

The next two examples demonstrate what can happen in problems 
which are not "normal" and where ker (E^) and ker (B^) are 
complementary spaces (see section 2) „ For these examples the 
attainable sets at time t can be determined without difficulty, 
enabling one to make a judicious choice (whenever there is more than 
one support hyperplane at the boundary point) of an outward normal 
for use directly in the maximum principle,. 


Example 7°S * This is an example of the form P^ 0 The system equa- 
tions are 


(7°8) x = u(t), y = u(t-l). 

with boundary conditions the same as in equation (7«2) » For system 
(7„8) the domain of null controllability (here U = [-1,1]) 
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turns out to be 


®o = 'Vo '.'' 2 


|x -y I ^ 21 
1 0 ^O 1 J 


Thus the problem P x has a solution only if (^ 0 ^ 0 ) e ^ 0 ° 0n 
the other hand if (x Q ,y 0 ) e ^ 0 } then The ° rem ^° 2 a3sures us that 
problem P-^ has a solution . The attainable set at time t turns 
out to be ( x 0 ,-y 0 ) + wh ich we denote by J^(x 0 ,y Q ) and 

this can be explicitly computed: 




[x-x 0 | g t, |y-y 0 j ^ t, Jx-yi £ 2), 


Figure 10 shows * ~ ^ or t^ < 1 < t^« 

x 


Figure 10. 

Taking advantage of the simple geometric structure of ..^(t) 

one finds that ^Xq^q) = ma; x £| x o!> ^ X 0^ y 0^ 6 ^ 0° 

an admissible control (u^t}^ t = t(xQ^y^) satisfying the boundary 
conditions (7.2) is a time optimal solution. The maximum principle 
for' this problem says that if T] = (n^T] 2 ) ^ 0 is a vector which is 
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an outward normal to a support hyperplane for - J2^r(x ,y ) passing 

v O' 7 0 

through (0,0) and if t < 1, then 


(7.9) u(t) 


sgn [n], 


-1 g t g t - 1 


undetermined, 

1 

sgn [ T) ], 


t - 1 < t < 0 
OStit 


and if t k 1, then 


(7o9') 


u(t) = 


sgn [tj ], -1 g t < 0 

sgn [An ], 0 g t < t - 1 


1. 


sgn [n ], 


t - 1 g t g t 


Let us consider some of the possibilities. Suppose (x ,y ) is 

on the line y = x - 2 and y Q > -x QO Figure 11(a) shows how 

1 2 

J3£Yx^,y\.) is positioned at (0,0) and we see that n = ( r i ,n ) 

4 - 0 0 . 7 

v 1 2 12 ',, 
can he chosen so that n < 0 < n and |n I > n~° Using this n 

in (7„9' ) one obtains 


u(t) 


+1, -1 g t < 0 

-1, 0 g t g t 


if ts 1, with an obvious modification using (7„9) if t < 1 



6k 


Figure 11 „ 


If 


(vV e 


then Jtfjx-Qi y 0 ) 


^ - T o > - x o> x o- 2 <J o < V x o > ^ 

is positioned at the origin as shorn in 


— 11 
Figure 11(h), -where t = x Q > 1. Hence q = (q ,o), n <0 and 

( 7 « 9 ‘) gives no information on the interval [- 1 , 0 ) hut ( 7 . 9 * ) 

does specify u(t ) = -1, 0 ^ t S t ( In this situation it turns out 

that any u such that (u, t} is admissible in satisfying 

u(t) = - 1 , 0 ^ t I t, and which drives ( x q}^q) 1 ° (x -l,x^-l) 

at time t = 1, turns out to he optimal., Let (x,y) denote a 

response initiating at (x Q ,y 0 ) to a control {u,t} of the above 

form,, Then we see that x(t) = y(t), II t and 


(7.10) jy(t)|/]x(t)j ^ 1 

for 0 ^ t ^ to On the other hand if y^ > -x^, - 2 < y^ < x , 

and x Q ^ l(t ^ 1 ), then ( 7 . 10 ) is all that is required of the ad- 
missible trajectory (x,y) as long as the boundary conditions ( 7 . 2 ) 


are satisfied,, 
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Suppose now that x^ = y_ and y > 0„ Then we find that 

0 0 0 

t = y and 

u(t) = -1 -1 g t g t, 

is the optimal control. 

Xq and y^ < 0, then the optimal control {u ; t} is 

Utlt-1 

ostst 

where t - |y Q ] S 1. 

Using s imi lar techniques one obtains optimal controls [u, t} for 
ad 1 (x 0 ,y Q ) lying in with y^ 1 -x^„ By taking advantage of 

the symmetry with respect to the origin an optimal control can then 
be determined for (x^y^) in the remainder of 

figure 12 illustrates the typical situations » In this figure 
heavy lines indicate pieces of optimal trajectories when (u,t] is 
unique, and the broken lines indicate segments of optimal trajectories 
where the 'uniqueness of the optimal control does not obtain. „ 

In this problem & (t) is increasing but not expanding. 
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Figure 12. 0 

Example 7„4 0 In this example we look "briefly at the same situation 
as in Example 7.3 except we change to a problem of type P g where 
= 0. Eow the domain of null controllability (with U = [-1,1]) 
is 

9 q ( o ) = t(x 0 ,y 0 ) eR2 | ! x 0 “ y J " 

and the attainable set at time t which we again denote by 
-^V y o ) is equal to (Xg,y^) + ^ g (t) „ It is easily sham that 

-^( x o> y o) = {(x ^ y) £ r2 I x “ x ol - ^ l y ~ y o! S " x > ! x " y l S 1} 

for til and 

J^ t (v y 0 ) = e r2 l x “ x o! - ^ y = y o } 

•for 0 < t < 1. These sets are shown in Figure 13. 
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Figure 13o 

Again^ if one takes advantage of the simple geometry present in 
the problem, then the "minimum time" t is determined to be 


■ 


1+ iy 0 '> 


I.V 


(V y o> 5 *$1°), y 0 / o 


> Vo 1 e 0 o (o) ' y o “ °> 


which is discontinuous on ■ 3^(0) at every point on the line 


! X 0 I < 1} Y o = 0}o If e 9 o (0) > then 3X1 


€ E 

optimal control [u^t] exists for problem and t = t (x^y^) 

(Theorem 5» 2 ) Q The necessary conditions for this example are the 
same as in Example 7 <,3 (equations (7<,9) and (7<.9' )) except the con- 
dition on u(t),, -1 ^ t < 0 is deleted,, To solve this problem one 
considers (as in Example 7„3) the possible r| = (t} , h ) which are 

normal to support hyperplanes for _&f_(x ,y ) through (0,0) and 

t 0 0 

makes an appropriate choice when there is more than one candidate „ 
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We now consider some of the cases. If 3 1 (i.e., t s 2 ) } 

then (f.9 1 ) yields 

(7.11) u(t) = OStsT-1. 

If in addition to y Q ^ 1 we hare y^ = x Q - 1^ then u(t) - -1 ; 

0 g t g t and the optimal- control {u,t} is unique. On the other 
hand if y^ S 1 and y^ = x^ + 1, then in addition to (7.11) we 
find that u(t) = -i-l, t - 1 < t g t and again the optimal control 
{u,t} is unique. Wow if y Q a 1 and x Q - 1 < y Q < x Q + 1, then 
any admissible control u will he optimal as long as it satisfies 
(7.11) and is defined on [t~l ; t] so that the boundary conditions 
(7.2) are satisfied. The cases that we have just discussed are 
sham in Figure l4 (where again non-unique segments of optimal tra- 
jectories are denoted by broken lines) by the trajectories initiating 
at points p_. * p g ; and respectively. We note that in 

many cases the optimal trajectories contain subarcs which lie out- 
side the domain of null controllability. 


Figure l4. 
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If y Q = 0, then one can show that 

u(t) = -1, * 0 fi t S t if 1 S x Q > 0 

and 

u(t) = +1, 0 I t S t if -1 ^ x Q < 0, 

so that the optimal control is also unique and the corresponding 
trajectories are very simple. Finally, we consider one other typical 
situation when optimal controls u are not unique. Suppose 
0 < y Q < 1 and x Q - 1 <0 y Q < x Q . The necessary conditions still 
give (7.II), hut in this case any admissible u satisfying (7dl) 
and the boundary conditions (7°2) at time t = 1 + y^ < 2 is 
optimal. For example the trajectory Issuing from point p 2 in 
Figure 15 shows one of the many optimal trajectories starting at 
this point at time 0. This trajectory passes through q^ at time 
t - 1, • arrives at r 2 at time 1, passes through s 2 at some 
time t, 1 < t < t and finally arrives at 0 at time t. 

Other optimal trajectories are also illustrated in Figure 15 . 

In this figure once again heavy lines denote pieces of optimal tra- 
jectories where the optimal control is unique, while along broken 
lines the optimal control is not unique. 

In this example ^g(t) turns out to be increasing but not 


expanding 



70 


Figure 15, 


Finally, it is noted that if we consider system (7„8) with a 
problem of type then the domain of null controllability is 

merely the straight line y = x 0 This problem is easily solved and 
some optimal trajectories for this problem are depicted in 
Figure 16, 


Figure 16. 
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8 0 Delayed Control' problems and Dynami'c' Programming 

Consider once again Example 7»^ above. If we consider the 
optimal trajectory emanating from in Figure lU, we notice that 

this trajectory has subarcs which are not optimal. Thus the 
principle of optimality in its usual form [ 19 ] does not hold here 0 
This is not too surprising since this principle fails even in 
ordinary control problems with time dependent restraint sets U(t) 
if one interprets "state” to mean x(t) instead of (t,x(t)) 

(cf„ [ l6 ] ) 0 ■ However, in the problems we are studying this difficulty 
is more serious. 

We also observe that in Examples 7»1 and 7°2 the principle of 
optimality in its usual sense fails to be true. On the basis of 
this experience one expects the failure of this principle of 
optimality to be an intrinsic property of optimization problems in- 
volving systems of the form _9 ^(A,Bq, B^) and not just a peculiar 
property redounding from the special boundary conditions in 
Examples 7.2 and 7.4 or the particular criterion for optimality. 

Hence one anticipates serious obstacles to obtaining results for 

/ 

problems involving using dynamic programming, 

nonetheless, for certain special performance -indices we are able to 
adapt the methods of dynamic programming to problems governed by 
systems (A, B Q , B-, ) , even though it is easy to construct examples 
shot-ring that the standard principle of optimality is also invalid 
for these problems. 

. The remarks below are valid for time varying systems even though 
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■we shall, in keeping with our practice in this paper, restrict our 
presentation to the case of constant coefficients. 

Let cr! R n -» E and Lt R K R m -> K he given C 2 functions. 
Suppose U C R m is given and ^ eR is fixed. For t < t^ h 

we shall denote by II the problem, of minimizing 

*1 

J(u,° t 0 ,x Q ) = a (x (t 1 ) ) + / L(t,u(t))dt 

t o 

over the class of Sf -admissible controls u where x(* ' w u) 
is the solution of 5^ (A, (v is a given fixed function 

throughout) subject to x(t^) = x^. 

Remark 8.1. We shall consider only the free endpoint problem* 
problems with restricted endpoints x(t- ) e Z C R n require the 

usual modifications [3, 19 ] « 

1 2 
An easy calculation shows that the response to 

satisfies 

A 

whenever t Q g t^ - h, where x is the solution to 
(8.1) x(t) = Ax(t) + fl(t,t 1 )u(t) t e C*fc Q , t x 3 

subject to x(t Q ) = Tx q with 5x Q = x Q + / e‘ ■ ( ^ h)A B 1 v 0 (i).d| and 
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B 


0 


B q + e-% 


t e 

t < tj _ - h 


We shall denote "by H the problem, of min imi zing 

\ 

J (uj t 0 ,x Q ) s a(x(j, r j) + / L(t,u(t))dt 

t 

0 

over all bounded measurable controls u; [t^ t^] -» U where 
t Q < and x(-“ is the solution to (8.1) subject to 

A /\ 

x(t Q ) = x Q . Note that the payoff J • ( J ) depends only on 

x(t^) ( x^)) and not on x(t) ( x(t)) for .t < t^ 

A 

Since, J(u* t Q ,x Q ) = J(u; t^Tx^) for every t < t^ - h and 

n ^ 

Xq e R } we see that the problems' IE and II are equivalent when-' 

ever t Q ^ t^ - h. That is^ if;, for given initial data (I^Xq) 

A 

with t ^ t^ - h, u is optimal for then u^ -extended to 

[t -h^t ] by taking u - v is optimal for H with initial data 
-1 *0 ° 

(t Q ,T“ x Q ). Conversely, if u is optimal for H with initial data 
(■fc 0 ^ x 0 )^ i>o — ^i - k, then u restricted to [t^t^] is optimal for 

A 

n with initial data (t Q; Tx^) „ 

Applying the methods of dynamic programming to the problem fi 
we obtain the Kami It on- Jacobi equation [ 3 , 19] 

S (t ; z) + min (L(t,w) -f $ (t,z)f(t.,z,,w)} = 0 
well 


( 8 . 2 ) 



for t < t and z e R n ; where $(t, z) = inf J(u; t^z) and 
L ■ u 

f'(t,z,w) = Az + ftC^t^w. Solving (8.2) with data 3> (t^ z) = cr(z), 
one obtains $(t o ^x Q ) for t < t^x e R n . Since for 

Iq ^ ti - h we have S^t^x^) = ^C^q, H^q), w ^ ers ®(^i Z ) = i n - "t, 2 

one thus has the optimal payoff for problem H. It should be noted that 
although (8.2) is valid for t < rt ,, one has $(t, z) = $(t,Tz) only for 

• 1 - x /S 

t ^ -t ^ - h. In case v^ = 0, one has Tz = z and $(t,z) = $(t^z) 
for all t < t-^. 


A 


Let us now consider a special case of the problems 1I ; II for 
which (8.2) can be solved using known techniques. Denote by II 

Si 

A A 


and H respectively the problems II and H for quadratic payoffs 

Q. 

cr(x) = xsx, L(s,u) = uR(s)u where U = R m . We assume that S e ^ nr) 

i 

is symmetric positive semi -definite and R(s) e £ is symmetric 
positive definite for s e R . Application of known results to the 


problem 


n. 

a 


yields the optimal (feedback) control 


(8.5) u(t) = -R" 1 (t)J2*(t J t 1 )G(t)S(t) 


fjor t 


5 ivy 


where G satisfies the matrix Riccati equation 


- 1 , 


(8.4) G(t) + G(t)A + A*G(t) - G(t)fl(t,t 1 )R”- L (t)G*(t,t 1 )G(t) = 0 


for t e with boundary condition G(t^) = S„ Note that 

(8.5) gives a feedback solution for the problem 'H which can be 

used to solve the problem IT in the following manner. Given 

<1 
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(t 0 ,x Q ), t Q g - h, as initial data for the problem one solves 

A 

the problem with initial data (t Q ,Ix ), obtaining a feedback 

of the form (8.5) » Next one uses this in (8.1) to find the optimal 
xj i 0 e„, one solves 


(8.5) x(t) = (A-fi(t,t 1 )R“ 1 (t)fl x '(t,t 1 )G(t)}x(t) 


for t e [t^tf] with data x(t Q ) = Tx , Using this together with 
(8.5) gives the optimal open loop control for IT . 

M. 



and u. = v to find the optimal trajectory for problem H This 
*0 0 • q 
latter step is not necessary to find the. optimal value of the payoff 

for E since knowledge of x and u yields J(u° t x ) at once 

SL U U 

from 


t 0 ,x ) = J(u; t lx : ) = x(t )Sx(t ) + / u(t)H(t)u(t)dt. 

*0 


We note that in (8.1) and the performance index j(uj ^q; x q) we - 
could make the change of variable y = e'^x, and then system (8.1) 

o 

takes the form y = ft(t,t^)u(t) „ If one carries out these sub- 
stitutions, then the corresponding Riccati equation will have the 
simple form G - GC(t, t 1 )G = 0 which can often be solved by a 
quadrature (see [ 25 , p„ 227 ]). 


Remark 8,,2 0 It is not difficult to give a rigorous derivation 
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(including existence of the required solution to the Pdccati equa- 
tion ( 8 . 4 ) on the entire interval [t^ t^] ) of the above solution 

A 

to the problem II using the maximum principle for H and 

q. q. 

arguments similar to those by Lee and Markus [19, sections 3.2 and 

3 . 3 ] = 

Remark 8 . 3 . The above ideas can be applied to certain optimal con- 
trol problems where retardations occur in both the state and control 
variables. For the corresponding quadratic payoff problem H . one 

Si. 

can then use recent extensions of the Riccati theory [1, 8, 15, 17, 

27 ]. 
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